Preprocessor step

The Preprocessor step allows the application to modify the incoming data prior to executing the rest of the extraction workflow through a number of preprocessors. It also lets you define properties, to be added to each record or to the data as a whole. For instance, a unique ID could be created to be added to each record in the output for integrity checks later on, or a time stamp could be added to create reports. A tag could be added to process certain records differently. A preprocessor could remove certain records altogether.

One example of how a preprocessor could be used is given in a How-to: Using Preprocessors in DataMapper.

Properties

To add a property:

  1. Select the Preprocessor step on the Steps pane.
  2. On the Step properties pane, under Properties, click the Add button . See Properties for an explanation of the settings for properties.

To set the value of a property later on in the data mapping workflow, you can use an Action step (see Action step).

Preprocessors

The Preprocessor step can contain any number of preprocessors. They will be run in sequence before the data is sent to the data mapping workflow.

Note that preprocessors are not executed automatically while designing the data mapping workflow; you must therefore execute them manually. The reason for this is that preprocessors can potentially be quite lengthy operations that would hinder the automatic refresh of the display whenever anything is changed in the data mapping workflow.

To add a preprocessor:

  1. Select the Preprocessor step on the Steps pane.
  2. On the Step properties pane, under Preprocessor, click the Add button .
  3. Under Preprocessor definition, add the script. Preprocessing tasks must be written in JavaScript (see Using scripts in the DataMapper and DataMapper Scripts API).

Configuring the Preprocessor step

For an explanation of the settings for preprocessors, see: Preprocessor step properties.